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(54) Browser-based electronic messaging 

(57) In a computerized method tor messaging, a 
Web page includes an embedded tag specifying a mes- 
sage extension program. 

The Web page is requested from a Web server by a 
Web browser of a first client computer connected to the 
Web server computer by a communications network 



The extension program corresponding to the message 
tag is retrieved. A multimedia message is recorded as 
digital signals in a file of the client computer using the 
extension program. The file is sent to a second client 
computer as an electronic mail message. 
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, Description 
FIELD OF THE INVENTION 

The present invention relates generally to electronic messaging, and more particularly to messaging using the 
Internet 

BACKGROUND OF THE INVENTION 

Recently a number of products have become available for sending and receiving voice messages using the Inter- 
net Protocol (IP). For example. Internet telephone software applications implement real-time synchronous two-way 
voice communications between users using Internet TCP/IP protocols. As a restriction, the users at both ends of the 
communications link need a suitably equipped PC including the same proprietary software. As with standard telephone 
communications, the person for whom the message is intended must be available to answer the call at the time the call 
is placed, otherwise two-way communications cannot take place. 

Some electronic mail products can asynchronously send non-textual information, e.g., voice messages, via the 
Internet For example, the voice message is first recorded in a file using a specialized application program. Then, the 
file is converted to a corresponding Multipurpose Internet Mail Extension (MIME) format. The voice message can be 
forwarded to the intended recipient by attaching the MIME encoded file to an outgoing message using some MIME ena- 
bled e-mail application program. 

Voice messages have also become a standard feature of many World-Wide-Web (WWW or Web) pages. Typically, 
an Internet user locates a Web page using a Web browsing application such as Netscape Navigator, or Microsoft Inter- 
net Explorer. Once a page has been selected, audio content associated with the page can be displayed by the browser 
after the page is delivered. In most cases, the audio message is a predetermined and embedded part of the Web page 
with no specific recipient indicated. 

The audio content can be delivered to the browsers in a number of ways, ranging from simple audio (.wav) file 
transfer to proprietary streaming technologies, such as in RealAudio. The transfer is one way, from content provider, 
e.g., the web server, to the client browser. 

It is desired to provide for the creation of multimedia content from within a Web browser, using a browser extension 
specified by a Web page, and its subsequent delivery to a remote destination. Furthermore, it is desired that the deliv- 
ery mechanism be based on standard, non-proprietary, Internet protocols, such that the intended recipient may receive 
and process such a message using any e-mail client program that supports these protocols. 

SUMMARY OF THE INVENTION 

In a computerized method for sending multimedia messages, a Web page includes a tag specifying a browser 
extension program, and a location of the extension program. The Web page is accessible by a Web server computer. 
The Web page is requested by a Web browser of a first client computer connected to the Web server computer by a 
communications network The extension program is retrieved from the location specified in the tag of the Web page, 
and extends the Web browser. 

Subsequently, the extension program is executed to record a multimedia message as digital signals in a file of the 
client computer. The file can be sent to a second client computer as an encoded electronic mail message. 

The invention, in its broad form, resides in a computerized method and apparatus for sending a multimedia mes- 
sage, as recited in claims 1 and 8 respectively. 

As described hereinafter, an audio message maybe recorded using a microphone and a sound card; alternatively, 
the message may be recorded using a camera and a video card. 

BRIEF DESCRIPTION OF THE DRAWINGS 

A more detailed understanding of the invention may be had from the following description of a preferred embodi- 
ment, given by way of example and to be understood in conjunction with the accompanying drawing wherein: 

♦ Figure 1 is a block diagram of an electronic messaging system according to a preferred embodiment of the inven- 
tion; 

♦ Figure 2 is a block diagram of a Web page including an embedded tag specifying a messaging plug-in browser 
extension program; 
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♦ • Figure 3 is a flow diagram of a process to install a message extension program in a browser application program; 

♦ Figures 4A-4B are diagrams of a graphic user interfaces used by the browser extension program; 
5 4 Figure 5 is a flow diagram of a process for recording an audio message; 

♦ Figure 6 is flow diagram of a process to send a file including the message; and 
4 Figure 7 is a diagram of a graphic user interface of a context menu. 

10 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

Figure 1 shows a messaging system 100 according to a preferred embodiment of the invention. The system 100 
includes client computers 1 1 0, 1 20, and 1 30, and a server computer 1 40 connected to each other by a communications 
is network 150. 

The client computers 110, 120, and 130 can be portable computers, e.g., lap-tops, PCs, workstations, or larger 
computer systems. The server computer 140 is usually configured as a workstation, or larger computer. The computers 
are standardly configured with dynamic random access memories (DRAM) connected to one or more processors via a 
bus. The computers can include input/output devices (I/O) 111 such as a mouse, keyboard, display terminal, printer, 
20 and network interface. The systems can also include local storage such as floppies and hard disk drives 1 1 2 for persist- 
ently storing programs and data. 

Application programs can execute under the control of system software such as Microsoft Windows 95, or NT O/S. Pro- 
grams and data can also be loaded into the computers via the network 1 50. 

The client computer 110 includes a microphone 116 connected to a soundboard 117. The soundboard 117 can 

25 convert analog acoustic signals 102 to digital signals. The client computer 120 includes application programs for gen- 
erating World-Wide-Web (WWW) pages, e.g., page 200, described in further detail below. The client computer 130 
includes a standard MIME enabled electronic mail (e-mail) application, for example Microsoft Exchange. The client 130 
also includes a speaker 1 1 8 connected to a soundboard 1 1 9. The soundboard 119 can convert digital signals to analog 
acoustic signals 103. The server 140 includes Web server software for delivering Web pages, such as page 200, to 

30 requesting client computers. In the preferred embodiment, the network 150 uses internet TCP/IP communications pro- 
tocols at the logical levels. 

During operation of the system 100, a user of the client computer 110 activates a browser application program 101 
using, for example, the mouse. The browser 101 can be stored on the disk 112 of the client 110. For example, the 
browser 101 can be the Netscape Navigator, or the Microsoft Internet Explorer. The browser 101 is used for locating 

35 and requesting Web pages, such as page 200. The Web page 200 is identified by its unique addresses known as Uni- 
versal Resource Locators (URL) 201, for example, ^vww.digital.corn/user/my-home-page.html. ,, 

The user of the client computer 120 can create the page 200 using the HyperText Markup Language (HTML). The 
page 200 can be posted on the server 140 for general access by any of the users of the network 150. Alternatively, the 
page 200 can be stored on the disk 1 1 2 of the client computer 1 20 in a directory which is known to the server 1 40 as a 

40 repository of Web pages. 

The server 1 40 executes "httpd" web server software to deliver Web pages to client computers using the HyperText 
Transport Protocol (HTTP). For example, using the browser 1 01 , the user of the client 1 1 0 can access the page 200 by 
specifying the URL 201 of the page 200 in a request messages such as ^vww.digital.conv\iser/i^^ 
In response, the server 1 40 delivers the page 200 to the client 1 1 0. Once the page 200 arrives at the client 1 1 0, it can 

45 be stored locally on the disk 112, and/or viewed on an a display terminal using the browser 101. 

The web page 200 includes an embedded HTML tag 21 0. The tag 21 0 specifies parameters for a "browser exten- 
sion." A browser extension is a program which extends the capabilities of the browser application program 101. The 
specified extension can dynamically be retrieved by the client computer 1 1 0 via the network 150 and stored in the mem- 
ory of the client. The extension is executed from within the application environment of the browser. However, the exten- 

50 sion cannot run as stand-alone application software. 

In the preferred embodiment of the invention, the extension can be a "plug-in," or an "applet." A plug-in extension 
is a hardware/software platform-specific program written in native executable code. A plug-in can essentially do any- 
thing the browser can do. A plug-in is loaded and permanently installed in the client 1 10. An applet extension is a plat- 
form-independent program written in JAVA code, and interpreted by the JAVA virtual machine (JVM). Security features 

55 limit the capabilities of the applet browser extension. The code of the applet is temporarily loaded along with the page 
200. The code of the JAVA applet is discarded with the page. 

Figure 2 generally shows the form of a tag 210 specifying a browser extension in the form of a plug-in. Here, as 
shown in Rgure 2, the tag 21 C of the page 200 includes field 221 -224. The field 221 specifies a MIME "type" of the plug- 
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in. In this case, the standard type is "application", and an "extra" (x) user defined sub-type is "x-voiceme-plugin." The 
field 222 specifies the alignment and dimensions of the display "window" within which the graphic user interface (GUI) 
of the extension is to operate. The field 223 specifies a default e-mail address which is to receive messages generated 
by the extension. The field 224 specifies the location where the extension is stored. 

If the MIME type is not known to the internal table of the browser 101 . then the user of the browser can retrieve the 
extension from the location specified in field 224, and install the extension in the browser 1 01 . The location can be any 
computer accessible via the network 1 50. 

The HTML code for a comparable tag specifying an applet extension may be specified, in part, as the code in Table 



Table 1 
(title JVoiceme Applet (/title) 
<h1 >VoicemeApplet 1 <h1 > 

(applet code="VoicemeApplet.class" width=210 height=200) 
(a href = "VoicemeApplet.java">VoiceMelv'a> 

Figure 3 generally shows the processing of the browser 101 with respect to extensions. In step 305, the page 200 
is displayed. In step 310. the browser makes a determination if the page 200 includes a tag specifying an extension, 
step 310. If a page does not include an extension tag, then the browser waits for the next user command, step 301. 

Otherwise, in step 320, the browser 101 makes a determination if the browser extension is an applet or plug-in. In 
the case of an applet the extension can be fetched and executed immediately. In the case of a plug-in, a check is made 
to see if the extension (e.g. ( the dynamic link library (DLL) corresponding to the type 221 has already been installed by 
consulting its internal table. If the extension has been installed, then continue with step 510 of Figure 5. 

If the extension has not been installed, then display a plug-in installation icon in step 330. This is a signal to the user 
that the MIME type required to process the page 200 is not recognized. In step 340, determine if the user requests 
installation of the extension, or not. If the user does not request installation, continue as for step 301 . 

Otherwise, in step 350, retrieve the extension from the location specified in field 224 of the tag 210, and install the 
DLL for the extension in step 360. The user must exit and restart the browser 1 01 in step 370 so that the requisite MIME 
type, e.g., "applicationfc-voice-plugin" is recognized. In step 380, the page is requested again, so the process can now 
proceed with step 501 of Figure 5. 

As stated above, it should be noted, in the case of an applet extension, the applet is automatically downloaded into 
the client computer 110. The applet extension is discarded with the page. 

The operation of the message extension is now described with reference to Figures 4A-4B, and a process 500 of 
Figure 5. Figures 4A, 4B shows graphic user interfaces (GUI) which will be displayed in the window specified in field 
222 of the tag 210. The GUI 400 includes a record button 410 depicting a microphone icon, and a default "To:" field 420, 
e.g., as specified in field 223 of tag 210, and a volume field 430. 

The user of the browser 101 "clicks" on the record button 410 to activate recording, step 510 of Figure 5. The rela- 
tive volume is indicated by a sliding bar 431 during the recording step 520. The signal is processed by the soundboard 
1 1 7, and the digital signal is stored as a temporary file 1 1 3 on the disk 1 1 2 of the client conputer 1 1 0. 

Clicking on the record button 410 again stops recording, step 530. At this point the GUI 440 of Figure 4B is dis- 
played in the extension window. The GUI 440 includes Send, Play, and Cancel buttons 441-443, as well as the default 
"To:" field 420. 

Clicking on the Send button 441 transmits the temporary file 113 to the address(es) indicated in the field 420 as a 
MIME encoded file using the Internet "smtp" protocol. Sending the file 1 13 involves the sub-steps of a process 600 of 
Figure 6. Read the file 1 13 from the disk 119, step 610. Encode the file using, for example, a Base64 Content-transfer- 
Encoding , see RFC 1521, N. Bernstein, N. Freed, MIME (Multipur pose Internet Mail Extension^ Part one: Mechanism 
for Specifying and describing the format of Internet Mes sage Bodies. 09/23/1993, step 620. Wrap the encoded file with 
a mail header which include the address information, and a MIME type indicator, e.g., "audio/x-wav," step 630. Lastly, 
post the wrapped file with the command "mailto: (recipient® step 640, where recipient® are the addressees. 

Pressing the Play button 442 plays back the recorded message from the temporary file 1 13, step 542. The GUI 440 
does not change state during play-back. Clicking on the cancel button 443, step 543, deletes the temporary file 113, 
and overlays the GU 1 400 in the window. 

Clicking the "right" button of the mouse at any time, shows the context menu GUI 700 of Rgure 7. The To: field 710 
can be used to change the default recipients) who are to receive the message. A Subject field 720 can be used to enter 
a default "subject" header associated wfti the message. A field 730, is the default X-URL field of the message, for 
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example "http*7/www.dec.c»nV-u8er/home.htmi." 

A confirm send button 740 can be used to toggle automatic sending or not. If confirmation is disabled, then the 
message is sent as soon as recording has been stopped. The message is sent using the current default values as they 
would be displayed in the context menu 700. If confirmation is enabled, this is the normal state, the message must be 
s sent by clicking on the Send button 441 . The About button 750 displays administrative information about the extension, 
such as date, version number, and the like. 

As an advantage, the sender of the message, e.g., the user of the client computer 1 10 does not need any special 
software other than a standard Web browser. All message preparation and processing takes place within the applica- 
tion environment of the browser. To enable messaging, the sender only needs to locate, using the browser, any Web 
10 page that is enabled with a extension tag as described herein. This can be the page 200 created by the user of the client 
120, or any other user, including the user of the client 1 10 or the client 130. 

The recipient of the message, e.g. , the user of the client 1 30, or any other user having a computer connected to the 
Internet, does not need any additional software other than a standard MIME enabled e-mail application. The choice of 
hardware and operating system is entirely independent of message extension as described herein. 
75 Although the invention has been described with respect to audio messaging, it should be understood that the mes- 
sage can also be configured to communicate using other multimedia modalities. For example, the microphone 1 1 6 and 
soundcard 1 1 7 can be replaced by a video camera and video card. In this case, the extensions type can be Image" and 
the message encoded in the temporary file 113 has the corresponding MIME type. Images will be recorded and 
encoded when clicking on the record button 410 of Figure 4, perhaps showing a camera icon. The recipient of the mes- 
20 sage will need a computer equipped with a video card to display the Image" MIME file. 

It should also be understood, that a client computer equipped with both audio and image devices can simultane- 
ously record sound and image data if both "audio" and "image" extensions are executed by the browser. In this case, 
the recipients of messages will receive messages having both sound and image signals. Furthermore, browser exten- 
sions can also be configured to perform text messaging, or still picture messaging, e.g., graphics. In the first case, click- 
25 ing on the record button 41 0 permits the entry of a text message with the extension. The message in this case is simply 
sent as a text" MIME type. 

The principles of the invention have been described with respect to specific embodiments. It will be apparent to 
those skilled in the art that various modifications can be made to the embodiments shown. 

30 Claims 

1 . A computerized method for sending multimedia messages, comprising: 

adding a tag for specifying an extension program to a page accessible by a server computer; 
35 requesting the page by a browser application executing in a first client computer connected to the server com- 

puter by a communications network; 
retrieving the extension program via the server computer; 

executing the extension program from the browser application program to record a multimedia message as dig- 
ital signals in a file of the first client computer; and 
40 sending the file to a second client computer as an encoded electronic mail message. 

2. The method of claim 1 further comprising: 

recording an audio message in the file using a microphone and a sound card. 

45 

3. The method of claim 1 further comprising: 

recording a video message in the file using a camera and a video card. 
so 4. The method of claim 1 further comprising: 

recording a still image message in the file using a camera and a frame grabber card. 
5. The method of claim 1 further comprising: 

55 

specifying a multipurpose internet mail extension (MIME) type, a default recipient address, and a location of 
the extension program in the embedded tag. 
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The method of claim 1 wherein the extension program is a browser plug-in. and further comprising: 

retrieving a dynamic link library corresponding to the extension program from the location in response to the 
Web browser processing the Web page. 

The method of claim 1 wherein the extension program is a browser applet, and further comprising: 

retrieving an applet corresponding to the extension program from the location in response to the Web browser 
processing the Web page. 

An apparatus for sending electronic messages, comprising: 

a server computer having access to a page including an embedded tag specifying a extension program; 
a first client computer, connected to the server computer by a communications network, requesting the page 
using a browser application program; 
means for retrieving the extension program specified in the tag; 

means for executing the extension program to record a multimedia message as digital signals in a file of the 
first client computer; 

means for sending the file to a second client computer as an encoded electronic mail message. 
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